Efficient concept formation in large state spaces

General autonomous agents must be able to operate in previously unseen worlds with large state spaces. To operate successfully in such worlds, the agents must maintain their own models of the environment, based on concept sets that are several orders of magnitude smaller. For adaptive agents, those concept sets cannot be fixed, but must adapt continuously to new situations. This, in turn, requires mechanisms for forming and preserving those concepts that are critical to successful decision-making, while removing others. In this paper we compare four general algorithms for learning and decision-making: (i) standard Q-learning, (ii) deep Q-learning, (iii) single-agent local Q-learning, and (iv) single-agent local Q-learning with improved concept formation rules. In an experiment with a state space larger than 232, it was found that a single-agent local Q-learning agent with improved concept formation rules performed substantially better than a similar agent with less sophisticated concept formation rules and slightly better than a deep Q-learning agent

Spatial complexity measure for characterising cellular automata generated 2D patterns

Cellular automata (CA) are known for their capacity to generate complex patterns through the local interaction of rules. Often the generated patterns, especially with multi-state two-dimensional CA, can exhibit interesting emergent behaviour. This paper addresses quantitative evaluation of spatial characteristics of CA generated patterns. It is suggested that the structural characteristics of two-dimensional (2D) CA patterns can be measured using mean information gain. This information-theoretic quantity, also known as conditional entropy, takes into account conditional and joint probabilities of cell states in a 2D plane. The effectiveness of the measure is shown in a series of experiments for multi-state 2D patterns generated by CA. The results of the experiments show that the measure is capable of distinguishing the structural characteristics including symmetry and randomness of 2D CA patterns

Avalanches in self-organized critical neural networks: A minimal model for the neural SOC universality class

The brain keeps its overall dynamics in a corridor of intermediate activity and it has been a long standing question what possible mechanism could achieve this task. Mechanisms from the field of statistical physics have long been suggesting that this homeostasis of brain activity could occur even without a central regulator, via self-organization on the level of neurons and their interactions, alone. Such physical mechanisms from the class of self-organized criticality exhibit characteristic dynamical signatures, similar to seismic activity related to earthquakes. Measurements of cortex rest activity showed first signs of dynamical signatures potentially pointing to self-organized critical dynamics in the brain. Indeed, recent more accurate measurements allowed for a detailed comparison with scaling theory of non-equilibrium critical phenomena, proving the existence of criticality in cortex dynamics. We here compare this new evaluation of cortex activity data to the predictions of the earliest physics spin model of self-organized critical neural networks. We find that the model matches with the recent experimental data and its interpretation in terms of dynamical signatures for criticality in the brain. The combination of signatures for criticality, power law distributions of avalanche sizes and durations, as well as a specific scaling relationship between anomalous exponents, defines a universality class characteristic of the particular critical phenomenon observed in the neural experiments. The spin model is a candidate for a minimal model of a self-organized critical adaptive network for the universality class of neural criticality. As a prototype model, it provides the background for models that include more biological details, yet share the same universality class characteristic of the homeostasis of activity in the brain.Comment: 17 pages, 5 figure

A self-organized model for cell-differentiation based on variations of molecular decay rates

Systemic properties of living cells are the result of molecular dynamics governed by so-called genetic regulatory networks (GRN). These networks capture all possible features of cells and are responsible for the immense levels of adaptation characteristic to living systems. At any point in time only small subsets of these networks are active. Any active subset of the GRN leads to the expression of particular sets of molecules (expression modes). The subsets of active networks change over time, leading to the observed complex dynamics of expression patterns. Understanding of this dynamics becomes increasingly important in systems biology and medicine. While the importance of transcription rates and catalytic interactions has been widely recognized in modeling genetic regulatory systems, the understanding of the role of degradation of biochemical agents (mRNA, protein) in regulatory dynamics remains limited. Recent experimental data suggests that there exists a functional relation between mRNA and protein decay rates and expression modes. In this paper we propose a model for the dynamics of successions of sequences of active subnetworks of the GRN. The model is able to reproduce key characteristics of molecular dynamics, including homeostasis, multi-stability, periodic dynamics, alternating activity, differentiability, and self-organized critical dynamics. Moreover the model allows to naturally understand the mechanism behind the relation between decay rates and expression modes. The model explains recent experimental observations that decay-rates (or turnovers) vary between differentiated tissue-classes at a general systemic level and highlights the role of intracellular decay rate control mechanisms in cell differentiation.Comment: 16 pages, 5 figure

Astrobiological Complexity with Probabilistic Cellular Automata

Search for extraterrestrial life and intelligence constitutes one of the major endeavors in science, but has yet been quantitatively modeled only rarely and in a cursory and superficial fashion. We argue that probabilistic cellular automata (PCA) represent the best quantitative framework for modeling astrobiological history of the Milky Way and its Galactic Habitable Zone. The relevant astrobiological parameters are to be modeled as the elements of the input probability matrix for the PCA kernel. With the underlying simplicity of the cellular automata constructs, this approach enables a quick analysis of large and ambiguous input parameters' space. We perform a simple clustering analysis of typical astrobiological histories and discuss the relevant boundary conditions of practical importance for planning and guiding actual empirical astrobiological and SETI projects. In addition to showing how the present framework is adaptable to more complex situations and updated observational databases from current and near-future space missions, we demonstrate how numerical results could offer a cautious rationale for continuation of practical SETI searches.Comment: 37 pages, 11 figures, 2 tables; added journal reference belo

Boolean Dynamics with Random Couplings

This paper reviews a class of generic dissipative dynamical systems called N-K models. In these models, the dynamics of N elements, defined as Boolean variables, develop step by step, clocked by a discrete time variable. Each of the N Boolean elements at a given time is given a value which depends upon K elements in the previous time step. We review the work of many authors on the behavior of the models, looking particularly at the structure and lengths of their cycles, the sizes of their basins of attraction, and the flow of information through the systems. In the limit of infinite N, there is a phase transition between a chaotic and an ordered phase, with a critical phase in between. We argue that the behavior of this system depends significantly on the topology of the network connections. If the elements are placed upon a lattice with dimension d, the system shows correlations related to the standard percolation or directed percolation phase transition on such a lattice. On the other hand, a very different behavior is seen in the Kauffman net in which all spins are equally likely to be coupled to a given spin. In this situation, coupling loops are mostly suppressed, and the behavior of the system is much more like that of a mean field theory. We also describe possible applications of the models to, for example, genetic networks, cell differentiation, evolution, democracy in social systems and neural networks.Comment: 69 pages, 16 figures, Submitted to Springer Applied Mathematical Sciences Serie

Menstrual irregularity and bone mass in premenopausal women: Cross-sectional associations with testosterone and SHBG

Background. There have been few studies examining the associations between menstrual irregularity, androgens and bone mass in population-based samples of premenopausal women. This study aimed to describe the associations between menstrual pattern, testosterone, sex hormone binding globulin (SHBG) and bone mass in a population-based sample of premenopausal women. Methods. Cross-sectional study (N = 382, mean age 31.5 years). Menstrual pattern was assessed by questionnaire, bone mass measured by quantitative ultrasound (QUS) and androgen status was assessed by levels of serum testosterone, SHBG and the free androgen index (FAI). Results. Women with irregular cycles (n = 41, 11%) had higher free androgen index (FAI, P = 0.01) and higher QUS measurements including speed of sound (SOS, 1%, P < 0.05), quantitative ultrasound index (QUI, 7%, p < 0.05), and broadband ultrasound attenuation (BUA, 7%, p = 0.10). These associations persisted after adjustment for age and body mass index (BMI). After further adjustment for hormonal factors (either testosterone, SHBG or FAI), the strength of the associations was moderately attenuated, however, women with irregular cycles still had a 6% increase in mean QUS. Total testosterone, FAI and SHBG were also associated with QUS measures (testosterone and FAI, r +0.11 to +0.21, all p < 0.05; SHBG r -0.14 to -0.16, all p < 0.05) and the associations remained significant after adjustment. Conclusion. Irregular menstrual cycles were associated with higher bone mass in this population-based sample of premenopausal women suggesting menstrual disturbance should continue to be evaluated but may be less harmful for bone mass. The association between menstrual irregularity and bone mass was partially mediated by markers of androgen status especially free testosterone